1. Field of the Invention
The present invention pertains to video data processing. More particularly, this invention relates to a system and a method for automatically detecting sharp and gradual shot boundaries and key frames from a compressed video data without fully decompressing the video data.
2. Description of the Related Art
With the advances in digital storage, Internet, and multimedia computing technologies, storing, accessing, and manipulating digital video are now becoming possible. For example, a video program or a collection of video programs can be stored by an optical disc and be accessed by a computer system connected to an optical disc driver with the optical disc either locally or via a network. This allows creation of a video image database that can store a collection of video programs.
However, one of the challenging problems in image or video databases is the organization of video information. Segmenting a video program into a number of appropriate units (i.e., clips) and characterizing each clip for indexing has been suggested as one possible mechanism for organizing video information. This approach requires that the cut points in the video program be located. One way of locating the cut points is to determine the boundaries between consecutive camera shots. As is known, a shot in video parlance refers to a contiguous recording of one or more video frames depicting a continuous action in time and space. During a shot, the camera may remain fixed, or it may exhibit one of the characteristic motionsxe2x80x94namely, panning, tilting, or tracking.
Determining the shot boundaries in a video has a number of practical applications. For example, the isolation of shots in video databases is of interest because the shot level organization of video documents is considered most appropriate for video browsing and content based retrieval. Shots also provide a convenient level for the study of the video program.
A number of prior methods have been proposed to automatically determine the shot boundaries in a video program. One such prior method is described in U.S. Pat. No. 5,635,982, issued on Jun. 3, 1997, by Hong J. Zhang, and entitled SYSTEM FOR AUTOMATIC VIDEO SEGMENTATION AND KEY FRAME EXTRACTION FOR VIDEO SEQUENCES HAVING BOTH SHARP AND GRADUAL TRANSITIONS. However, one disadvantage of this prior approach is that it is based on uncompressed video data. This means that the prior approach can only detect shot boundaries from uncompressed video data. Decompression is required if the video data is a compressed one. As more and more video programs are stored and transmitted in compressed digital format to reduce the data volume of the video programs for storage and transmission, the decompressed video data needs to be re-compressed after shot boundary detection. This is not cost effective because the decompression and compression are otherwise not needed. In addition, due to the need for decompression, the prior approach takes relatively longer time to complete the shot boundary detection.
Another prior approach to automatically detect shot boundaries in video data is described in U.S. Pat. No. 5,606,655, issued on Feb. 25, 1997, by Farshid Arman et al., and entitled METHOD FOR REPRESENTING CONTENTS OF A SINGLE VIDEO SHOT USING FRAMES, and U.S. Pat. No. 5,521,841, issued on May 28, 1996, by Farshid Arman et al., and entitled BROWSING CONTENTS OF A GIVEN VIDEO SEQUENCE. However, one disadvantage of this prior approach is that it typically cannot detect gradual transitions between shots. This means it cannot detect camera operations such as panning, zooming, or other special effects.
One feature of the present invention is to allow automatic detection and extraction of shot boundaries and key frames from a compressed video data without decompression.
Another feature of the present invention is to allow automatic detection and extraction of sharp and gradual shot boundaries, camera operations, and key frames from a compressed video data.
A still another feature of the present invention is to allow fast and robust automatic detection and extraction of shot boundaries and key frames from a compressed video data.
A further feature of the present invention is to provide an adaptive threshold selection scheme for the automatic detection and extraction of shot boundaries and key frames from the compressed data.
A system for detecting shot boundary in a compressed video data without decompressing the video data includes a difference detector that detects content difference between a current frame of a current group of frames and previous frames of previous groups of frames. When the difference detector determines that a first content difference between the current frame and a previous frame of a first previous group of frames is greater than a first threshold (TS), a sharp shot boundary detector is used to detect a sharp shot boundary between the current frame and the previous frame of the first previous group. When the difference detector determines that (1) the first content difference is not greater than the first threshold and (2) a second content difference between the current frame and a previous frame of a second previous group of frames is greater than a second threshold (TG), a gradual shot boundary detector is used to detect a gradual shot boundary between the current frame and the previous frame of the second previous group. The second previous group of frames are prior to the first previous group of frames.
The gradual shot boundary detector includes a pan frame detector that detects pan boundaries, a zoom detector that detects zoom boundaries, and key frame detector that detects key frames. In addition, the system also includes a threshold selection unit coupled to the difference detector to select the thresholds in accordance with the content difference detected such that the sharp shot boundary detector, the gradual shot boundary detector, and the key frame function efficiently and effectively.
Other features and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.